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ABSTRACT 

This report describes challenges, opportunities, and pitfalls 
that exist when attempting to replicate successful youth programs, focusing on 
the replication of a program originally developed by the Children’s Village, 
Dobbs Ferry, New York. The program was designed to motivate youth in the child 
welfare system's residential treatment program to stay in school and develop 
and practice sound work ethics after discharge from care. The program, "Work 
Appreciation for Youth, " was implemented by four community-based organizations 
with out-of-school youth living in their own communities. Examination of the 
replication process suggests a number of key issues for consideration by 
administrators and practitioners of replication sites, sponsoring agencies, 
and funders. Findings highlight the fact that identifying successful program 
models and promoting replications is only part of the challenge. Successful 
replications require careful consideration of several key issues by the 
replicating site, sponsoring agency, and funding agency (e.g., establishing a 
mutually agreed-upon quality control mechanism for replication sites before 
starting; ensuring broad, high-level organizational commitment to the program; 
developing an implementation manual; making sure programs are large enough; 
allowing for extended startup time; and facilitating ongoing funding) . (SM) 
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Preface and Acknowledgements 



T ihis paper provides insights about the replication of a successful 
program originally developed by The Children’s Village (CV), 
Dobbs Ferry, NY. The program was designed for youth in the child 
welfare system’s most restrictive level of care — residential 
treatment — to assist and motivate them to stay in school and to 
develop and practice sound work ethics after they are discharged from 
care. The program, “Work Appreciation for Youth” (WAY), was 
found to be successful for this population of youth and was included 
in a report of approximately 50 evaluations described in the American 
Youth Policy Forum’s (AYPF) 1999 publication, MORE Things That 
DO Make A Difference for Youth: A Compendium of Evaluations of 
Youth Programs and Practices. (This was the second of a series of 
AYPF compendia dedicated to informing policymakers and 
practitioners about “what works” to improve life prospects for youth.) 

Among the strategies that consistently surfaced across the over 100 
programs cited in AYPF compendia series was the importance of 
implementation quality. The review of the evaluations of successful 
programs found: 

Factors contributing to successful implementation are: ample 
start-up time; clear communication of goals; sufficient, timely 
and sustained resources; strong leadership from the federal, 
state or local levels; staff development; and use of data to 
improve performance. When evaluations show negative 
results, it is not always due to flaws in elements of the model, 
but rather to flaws of implementation. (1999, p. xi) 

When Nan Dale, Amy Baker and David Racine recorded their 
reflections on the replication of the WAY model, the American Youth 
Policy Forum thought it important to share their thinking with the field 
of youth-serving practitioners, researchers, evaluators and funders. 
Having captured many of the basic principles of effective youth 
programs in the original WAY model (such as the availability of caring, 
knowledgeable adults, high standards and expectations, holistic 
approaches to the needs of youth, work-based learning, and long-term 
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services/support and follow-up* ), we also thought it important for others 
to know how these strategies played out in the implementation of the 
replications at four sites. Hence, the genesis of this joint endeavor by 
The Children’s Village and the American Youth Policy Forum. 

A case study of this replication effort, however, should not be seen in 
isolation. There have been numerous replications of successful 
programs, which also have valuable tales to be told. With that in 
mind, we asked social policy analyst Tom Smith to provide a general 
background and history of replication efforts — both federally- and 
privately-supported— and the role that they have played in the 
expansion of youth programs nationwide. In the Foreword to this 
report, he provides insights into the role that evaluation research has 
played in increasing or diminishing funding for and replication of 
successful programs and offers sage counsel about the necessary 
preconditions for further replication efforts. 

AYPF thanks the WAY report authors, Nan Dale, Amy J.L. Baker and 
David Racine, whose concern rings clear throughout the case study 
for maintaining high standards of implementation sufficient to 
transform the lives of vulnerable youth beyond an individual program 
and into its replication. We are especially grateful that they have 
taken the time to record their experiences and share them so that 
others may profit from the lessons learned. We also thank Tom 
Smith whose vast experience in research and youth program 
evaluation was used to capture the history, background and 
conclusions that can be drawn from other replication efforts. In 
addition to our colleagues at the American Youth Policy Forum who 
have helped in the editing of this report, we also thank Rafael 
Chargel, a talented computer graphics artist, who has formatted the 
document for publication. 

Special appreciation also goes to the Ford Foundation whose 
continued support enables us to carry on an extended conversation 
and knowledge exchange with policymakers and practitioners working 
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* These are many of the basic principles reflected in the current 
federal Workforce Investment Act for youth employment and training 
programs. 
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on youth development issues at the local, state and national levels. 

This allows the American Youth Policy Forum, through our learning 

events and policy reports, to: 

• document the realities facing at-risk youth, their communities and 
the institutions and agencies that serve them; 

• communicate research findings on effective practices to a very 
broad audience; and 

• highlight issues of policy and practice that must be addressed if 
our nation is to realize improved outcomes for disadvantaged 
youth. 



— Betsy Brand and Glenda Partee, Co-Directors, 

American Youth Policy Forum 



Foreword: The Uncertain 
Enterprise of Replication 

F br about 30 years, social evaluators have sought to apply the 
rigorous standards of science, econometrics and statistics to 
human service programs for adults, adolescents and children. Their 
success has been considerable. Within the comparatively small 
community of academic researchers, social policy analysts, 
congressional and public agency staff who care about such issues, 
there is pretty fair agreement about what constitutes persuasive 
evidence of success, and which technical and methodological 
approaches yield such evidence. 

The evidence was to play two key roles. First, and most obviously, it 
should tell us “what works” with certainty. That in turn would 
generate support for social programs, persuading the detractors who 
otherwise would dismiss all such programs as ineffective. Second, 
the programs that work would be widely adopted', their success, to 
use that much overused word, would be replicated. 

As with all such human enterprise, there have been failures and 
unexpected consequences. In the late 1970s, for example, the Youth 
Employment Demonstration Projects Act spent over $700 million in 
an effort to establish, through systematic research, which 
programmatic approaches “worked.” Much of the research went 
uncompleted, the quality of other work was later questioned, and 
most of the programs created under the Act have long since vanished. 

Among unintended consequences, perhaps the most dismaying was the 
discovery of how harsh the effects of evaluation could be. Nothing 
exemplifies this better than the Job Training Partnership Act (JTPA) 
evaluation, completed at the beginning of the 1990s. Using the “gold- 
standard” methodology of random assignment, it found that youth 
employment programs had virtually no impact on the wages and earnings 

of participants. 

Specific program types were not identified. The “field was treated 
as a monolithic experience, and— as with much evaluation — the 
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results were broadcast in their most generic and damaging form. As 
a result, for more than a decade, federal youth employment programs 
suffered from a crisis of confidence and a gradual loss of funding. 



The path of replication has likewise been bumpy. In the early going 
it was quite naively assumed that replication was a more or less 
organic, automatic process. The natural “market” for quality 
programs would work without effort, and the program “winners” 
would spread on their own. 

Three things became apparent. 

• Evaluation research was slow and imprecise. It did not always 
identify clear winners, did not identify them quickly, or did not 
always identify them with enough detail to make their replication 
a simple matter. 

• The experience of social demonstration projects, whose task was 
to build program “replicas” in applied settings in order to test 
them, showed that replication was in fact far from easy or 
automatic. It took thinking, planning, negotiation, and 
continuous effort; it also took money. 

• The “market” for social programs did not always function 
efficiently. Some programs that had clearly “worked” failed in 
the replication phase. The issue of what constituted enough of 
the “right” evidence to certify a program’s effectiveness proved 
to be a blurry one in the marketplace of social programs. What 
got replicated too often proved to be not what worked, but rather 
what was aggressively marketed, or what best fit the current 
funding emphasis. 

The case study that follows, detailing the replication of the Children’s 
Village, “Work Appreciation for Youth” program, reflects much of the 
complexity to be found in both the subject and the reality of program 
replication. As a case study, it stands well on its own, as do the 
recommendations proposed by the authors. At the same time, it 
offers some useful opportunities to recall — and reflect upon — some 
of the broader history of replication in the youth employment field. 
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Hard Times 



Such reflection is timely, for the youth employment field faces a less than 
encouraging future. As the nation tightens its belt in the wake of the 
priorities emanating from the Sept 11 terrorist attacks, federal supports to 
the youth employment field appear jeopardized and there will likely be 
serious undermining of the Youth Opportunity Grants program the 
signature youth component of the Workforce Investment Act. After all, 
youth don’t vote, education is where the money will be spent (though not 
extravagantly there, either), and many of these youth programs “don’t 
work” anyway. 

Yet mixed in with the proposed cuts ties something of a surprise. The 
Job Corps, the oldest and most costly of all federal youth employment 
enterprises, is slated for a small increase, as it is viewed as one of the few 
cost-effective programs. 



The Job Corps is an exception among youth programs. It has compiled a 
track record strong enough to persuade even a skeptical Congress (and 
various skeptical presidential Administrations) to keep it intact. Its 
impact data and cost-benefit figures are impressive. And with over 120 
centers operating across the United States, Job Corps must also be 
judged a genuine replication success. Indeed, the successful operation of 
so many programs clearly has its role in convincing the skeptics that Job 
Corps is a program that works not just on paper but in the field as well. 

Undoubtedly, replication successes on the order of Job Corps would do 
much to raise confidence and support for the youth employment field in 
general. And a sympathetic look across that field would identify potential 
exemplars, such as youth service and conversation corps, Jobs for Youth 
and YouthBUILD. Like Job Corps, these programs have “replicated” 
with some measure of success. 

Admittedlytoo, we can find programs, apparently replication-worthy, 
that have failed to clear the hurdle. That history reminds us that there is 
still much to learn, and that some of our casual assumptions about 
replication need to be revisited. Three examples illustrate some of the 
key dimensions of the replication experience. 



Replication 1 : The Job Corps 



Since the Job Corps is plausibly a standard by which to judge replication, 
it is instructive to examine the list of Job Corps’ programmatic 
components. It is impressive, and — in thick, compendious operational 
manuals — impressively detailed: 

• Support for a one-year residential slot in a fully staffed facility 

• Life skills, counseling and other ongoing support services 

• Fully developed remedial curricula in basic subjects, usually 
computer-based 

• Occupational training in a wide array of industry areas, determined 
by the labor market of the individual site 

• Community service activities, including part-time employment 

• A highly developed accountability system that spans both cost and 
program outcomes. 

Each Job Corps site can draw on the support and experience of a 
national network. That network, in turn, is supported and managed by a 
national office that systematically sets and reviews performance 
standards and monitors their attainment by the local programs. One 
cannot fail to be impressed by the system, even recalling that Job Corps 
has had more than 30 years in which to work out the kinks. 

What has permitted the evolution of this sophisticated program and its 
supportive infrastructure is robust evidence of success — essential to the 
survival and growth of a program whose per-slot costs now range near 
$30,000 per year. What may have mattered more is that, befitting a 
program birthed and raised by a public agency, the key elements of Job 
Corps were, routinized and documented relatively early in the game — 
bureaucratized, in a positive sense. 

Job Corps, indeed, began its life as a program-in-replica. Unlike many 
youth programs, which came into being more by evolution and 
improvisation than by design and planning, and which depended heavily 



on unique local circumstances for design and operation, Job Corps began 
as a “designed” set of elements, was intended for implementation in 
numerous settings, and always had behind it the substantial staff and 
funding resources of a large federal agency. 1 

Replication 2: The Center for Employment and Training 

The replication history of the Center for Employment and Training 
(CET) in San Jose, California is far more typical of a replication effort 
supported by the federal government as a funder, and not as the direct 
owner or manager of the initiative. 

CET grew up in San Jose over a period of more than 20 years. Though 
it combined the classic job training components — skills training, 
education, personal support — CET was distinctive in three ways. First, 
it stressed consumer choice. Participants were encouraged to identify 
career pathways they wanted to pursue, so that their motivation to attend 
work and training would be high. 

Second, CET operated on the basis of “work first” as much as possible. 
Rather than starting with a potentially frustrating educational regimen, 
CET participants would often be placed directly in paying jobs, or 
immediately into a skills training environment. This was meant to 
enhance client motivation: participants came to the program wanting to 
work, and they got to work as quickly as possible. 

Most importandy, CET developed rich and robust ties to the local labor 
market. Over a twenty-year period, it grew into a labor market 
intermediary trusted by employers to provide dedicated employees who, 
if not workplace ready, were motivated to work and learn. And much of 
the trust was personal, resting on the relationship CET’s director had 
with individual business leaders. 



1 Editor’s note: Aside from matters of effective program design and 
implementation, Job Corps is supported by clearly the most 
powerful political network enjoyed by any youth program. Over the 
past 37 years, private profit-making firms have marshaled the clout 
of the business community on behalf of the national Job Corps 
program. Locally, pumping a total of a billion dollars annually into 
numerous Congressional districts, Job Corps has become a favorite 
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